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Appendix 

Proposal Probabilities for Model Adaptation. 

p $p (k,d) = c* ■ min{l,p/(fc + 1)}; 
Pme(k. d) = c ■ min{l. (k - l)/p}; 
Psw(k,d) = c*: 

d=l,...,D: 

Pem(k) = 1 - Z^iiPspik.d) + p me (k,d) +p sw (k,d)). 

Here, c is a simulation parameter, and k is the current number of states. The 
parameter p is the hyper-parameter for the truncated Poisson prior probability of 
the number of states, i.e., p is the expected mean of the number of states if the 
maximum state size is allowed to be +00 , and the scaling factor that multiplies c 
modulates the probability using the resulting state-space size k ± 1 and p . 

1 

Determining Different Moves in RJ-MCMC 

Expectation maximization (EM) is one regular hill-climbing iteration. After a 
move type other than EM is selected, one or two states at a certain level are 
selected at random for swap/split/merge, and the parameters are modified 
accordingly. 

Swap the association of two states: Choose two states from the same level, each 
of which belongs to a different higher-level state; swap their higher-level 
association. 
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Split a state: Choose a state at random. The split strategy differs when this state is 
at different position in the hierarchy: when this is a state at the lowest level (d = 
D), perturb the mean of its associated Gaussian observation distribution as follows 

A*x = yuo -+- li-sll 

where n s ~ U[0, 1], and 7 is a simulation parameter that ensures reversibility 
between split moves and merge moves. When this is a state at d = 1, ., D - 1, with 
more than one children states, split its children into two disjoint sets at random, 
generate a new sibling state at level d associated with the same parent as the 
selected state. Update the corresponding multi-level Markov chain parameters 
accordingly. 

Merge two states: Select two sibling states at level d, merge the observation 
probabilities or the corresponding child-HHMM of these two states, depending on 
which level they are located in the original HHMM: When d = D, merge the 
Gaussian observation probabilities by making the new mean as the average of the 
two. 

Mo = - — J 1 — , if U*i - M2| < 2/j 

When d= \, ...,D - \, merge the two states by making all the children of these two 
states the children of the merged state, and modify the multi-level transition 
probabilities accordingly. 

Acceptance Ratio for Different Moves in RJ-MCMC. 

The acceptance ratio for Swap simplifies into the posterior ratio because the 
dimension of the space does not change. Denote 0 as the old model and 0 as the 
new model: 
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r £ (posterior ratio) = Jg§j = 

When moves are proposed to a parameter space with different dimension, such as 
split or merge, we will need two additional terms in evaluating the acceptance 
ratio: a proposal ratio term to compensate for the probability that the current 
proposal is actually reached to ensure detailed balance; and a Jacobian term is used 
to align the two spaces. 

Here, p sp (k) and p ms (k) refer to the proposal probabilities, see above, with the extra 
variable d omitted because split or merge moves do not involve any change across 
levels. 
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